Stable Electromyographic Sequence Prediction During Movement Transitions using Temporal Convolutional Networks

Transient muscle movements influence the temporal structure of myoelectric signal patterns, often leading to unstable prediction behavior from movement-pattern classification methods. We show that temporal convolutional network sequential models leverage the myoelectric signal's history to discover contextual temporal features that aid in correctly predicting movement intentions, especially during interclass transitions. We demonstrate myoelectric classification using temporal convolutional networks to effect 3 simultaneous hand and wrist degrees-of-freedom in an experiment involving nine human-subjects. Temporal convolutional networks yield significant $(p<0.001)$ performance improvements over other state-of-the-art methods in terms of both classification accuracy and stability.Comment: 4 pages, 5 figures, accepted for Neural Engineering (NER) 2019 Conferenc

Coarse Electrocorticographic Decoding of Ipsilateral Reach in Patients with Brain Lesions

<div><p>In patients with unilateral upper limb paralysis from strokes and other brain lesions, strategies for functional recovery may eventually include brain-machine interfaces (BMIs) using control signals from residual sensorimotor systems in the damaged hemisphere. When voluntary movements of the contralateral limb are not possible due to brain pathology, initial training of such a BMI may require use of the unaffected ipsilateral limb. We conducted an offline investigation of the feasibility of decoding ipsilateral upper limb movements from electrocorticographic (ECoG) recordings in three patients with different lesions of sensorimotor systems associated with upper limb control. We found that the first principal component (PC) of unconstrained, naturalistic reaching movements of the upper limb could be decoded from ipsilateral ECoG using a linear model. ECoG signal features yielding the best decoding accuracy were different across subjects. Performance saturated with very few input features. Decoding performances of 0.77, 0.73, and 0.66 (median Pearson's <i>r</i> between the predicted and actual first PC of movement using nine signal features) were achieved in the three subjects. The performance achieved here with small numbers of electrodes and computationally simple decoding algorithms suggests that it may be possible to control a BMI using ECoG recorded from damaged sensorimotor brain systems.</p></div

Decoding model performance across sessions for the first PC of movement.

<p>Distributions are displayed for the five cross-validations of the performance for one, two, and nine input features. S1–1 stands for subject 1 session 1, S1–2 stands for subject 1 session 2, etc. The maximum of the 1024 chance decoding attempts for all five folds with shuffled neural data is shown with an asterisk.</p

Presurgical MRI and brain reconstructions.

<p>Reconstructions are shown for subject 1 (first row) subject 2 (bottom left and middle) and subject 3 (bottom right). The previous resection margins anterior to the precentral gyrus in subject 1 are highlighted in green in the upper right. Superior oblique and top axial views of the reconstruction for subject 2 show the lesion from different viewpoints. Pre-surgical MRI (FLAIR) of subject 3 reveals a lesion of posterior left insula also involving left internal capsule.</p

Correlation between actual and decoded kinematics with and without PCA.

<p>Same methods were employed as with the first PC. The 9 best correlated neural features were selected for decoding in each dimension. The median of five-folds of correlation is displayed for each session. Bold denotes a statistically significant difference from chance results (<i>p</i><0.05, Bonferroni-corrected Wilcoxon).</p><p>Correlation between actual and decoded kinematics with and without PCA.</p

Subject Demographic and Clinical Information.

<p>Subject Demographic and Clinical Information.</p

Anatomical patterns of average correlations between representative ECoG signal features and the first PC of the hand position.

<p>Correlations were calculated at each feature's best lag (within ±1 second) relative to the movement data. Color-coded circles corresponding to maximum correlations, averaged across experimental sessions, are shown for each subject.</p

Subject Kinematics and Experimental Paradigm.

<p>(A) Shows the weights given to each of the three dimensions (height, depth, and lateral) when computing the first PC of the movement kinematics. Each point represents the PC weight calculated using a different cross-validation. (B–D) Show different views of the experimental paradigm and example kinematics from one of the subjects' sessions. The red dashed line represents the first PC plotted in Cartesian space for that same session.</p

Reconstructions of electrode placements with ESM results.

<p>The grids shown are the subset of implanted electrodes that were recorded from during this study. The green highlighted areas correspond to regions of cortical lesions. The lesion in subject 3 could not be seen on the brain surface rendering because it was located beneath the surface of the brain. Colored rectangles joining electrodes imply that bipolar stimulation was applied to that pair of electrodes. In subject 1, several electrode pairs were further investigated by performing unipolar stimulation relative to a distant reference electrode. The results of this unipolar stimulation are shown with colored circles surrounding the electrodes.</p

Relationship between number of model inputs (i.e., ECoG signal features) and model performance.

<p>Mean decoding accuracies of the first PC of movement across cross-validations and sessions are displayed for features chosen in decreasing order of correlation. Shading corresponds to the 95^th percentile confidence interval for the mean.</p